1. Field of the Invention
The present invention relates to an automatic interpreting system, an automatic interpreting method, and a storage medium storing a program for automatic interpreting, for the purpose of interpreting conversation between speakers of different languages. In particular, the present invention relates to an automatic interpreting system, an automatic interpreting method, and a program for automatic interpretation that can provide guidance for recovery of communication between users if the interpretation of an utterance is not understandable to the receiving party.
2. Description of the Related Art
In order to achieve automatic interpretation of conversation between speakers of different languages, R&D activities have been promoted in the fields of individual underlying technologies, i.e., speech recognition, automatic translation, and speech synthesis, with focus on enabling audio input of an utterance and audio output of an interpretation of such utterance. Thanks to the advancements made in these fields, the performance of automatic interpretation has steadily improved in recent years, giving additional momentum to R&D activities toward the commercialization of automatic interpretation in and outside Japan. When put to practical use, the technology of automatic interpretation as described above is expected to be utilized by a wide variety of speakers who talk of many different topics in many different situations. It is thus critical for the technology to achieve very high performance consistently in whatever environments in which it may be utilized.
A number of approaches for resolving problems related to robustness have already been proposed. One of these approaches allows the user to identify an error in or a failure of speech recognition or translation and to correct it as necessary. Another approach features automatic detection of errors in recognition results and translation results, audio presentation of recognition results, and presentation of translation results obtained through reverse translation. All these approaches contribute to a reduction in labor on the part of the user in finding errors.
As for the former approach, one method is disclosed, for example, in Japanese Patent Laying-Open No. 09-258776, in which the user corrects errors through pre-edition of sentences for input for automatic translation. According to this method, the user sees errors directly and thus reliable removal of errors can be ensured.
As for the latter approach, one method is disclosed, for example, in Japanese Patent Laying-Open No. 06-203069, in which labor on the part of the user required for correcting errors is reduced by performing grammatical check on sentences before translation and then presenting any error found in the sentences and an action for correcting it.
In the area of automatic interpretation, various methods have been devised and proposed for determining whether recognition results or translation results are correct or not and then correcting detected errors. In fact, there exists a technique for automatic determination of the correctness of speech recognition results or translation results. These, however, are yet to achieve an adequate level of accuracy and coverage. Thus, many of the proposals made in relation to error correction rely on the speaker for final determination and correction of errors.
For example, if the user who is conversing with the aid of an interpreting mechanism is unable to understand an interpreted utterance of the other party, then the inability to understand is due to either an interpretation error in the current utterance or one in the previous utterance. In order to recover the flow of conversation or the conveyance of the meaning when such situation occurs, it is critical to determine the conversation status correctly and take an appropriate action.
However, if the user has little knowledge of the language used by the other party, or has little experience in utilizing an automatic interpreting system, the user can hardly be expected to determine correctly the conversation status and an action to take. Thus, conventional methods, which rely on the speakers for final determination of errors and corrective actions, holds the problem of limitation; they can be viable only if the speaker has some knowledge of the language used by the other party, along with some experience in utilizing an automatic interpreting mechanism.